6 ms·

I've noted before and will note again that the amount of hate Python got for the 2-to-3 deprecations has led to a much worse problem: piecemeal low-visibility u

I've noted before and will note again that the amount of hate Python got for the 2-to-3 deprecations has led to a much worse problem: piecemeal low-visibility updates which are much harder to plan for. As a contractor, I singlehandedly migrated 4 different codebases from 2 to 3; a total of just under a million lines of code, and none of these codebases took more than 40 hours. This was not some horrific burden for most companies using Python. The people who flamed Python for these breaking changes were either extreme outliers or the sort of people who would be happy with the language never making breaking changes, ever, no matter how bad of mistakes in design were discovered. I would strongly prefer that datetime.utcnow were not deprecated in a 3.x release, but instead deprecated along with the GIL and any other breaking changes, in Python 4.0. Yes, it would get them flamed. And the flamers are wrong. At least that way people can plan for the transition at their leisure instead of constantly worrying that a 3.x release will unexpectedly break their codebase if they don't read the release notes before every update.

dheera 3y ago
Deprecation shouldn't break peoples' code all of a sudden. Code should still work, and just get more annoying over time. Initially, spit out warnings that deprecation is going to happen. Then, start changing the user's desktop background. Then, start playing nice cat meowing sounds out of the speaker. Then, start playing ghastly wailing sounds out of the speaker. Then, start sending e-mails. Then, scan for IoT light bulbs and mess with the colors. Then, scan for bluetooth headsets within a 50 meter radius and play wailing noises out of those. Then, impose a 0.001 second delay. Then a 0.01 second delay. Then a 0.1 second delay. Then a 1.0 second delay. Then start scraping LinkedIn for engineer resumes and e-mail them to any sysadmin e-mails you can find in the various config files in /etc/ and /home/. Don't suddenly deprecate stuff and break people's code on a moment's notice. Most likely the person who wrote the code using the deprecated function was laid off 2 years ago and the company still depends on it.
- panarky 3y ago
  > Don't suddenly deprecate stuff and break people's code on a moment's notice. "Deprecated" doesn't mean "break people's code on a moment's notice". FTA: "datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version" "likely a few years out" "once you switch to Python 3.12 or newer you will start seeing deprecation messages on your console and your logs, and these can get annoying"
  - dheera 3y ago
    > will be removed in a future version It should not be "removed" as a step function. It should get more annoying over time but still function for the next 30 years, at which point it's incredibly, incredibly annoying to call the function (but still works). This helps products continue running but simultaneously incentivizes the people on top to hire engineers to make the code less annoying without completely wrecking the business or downstream software that depends on it. Basically, the servers will still run and the customers will be happy but if the VP sitting 50m from the servers doesn't want cat noises coming out of their headphones they need to hire an engineer ASAP or it will turn into wailing noises pretty soon. > you will start seeing deprecation messages on your console and your logs, and these can get annoying Not annoying enough. This only sends messages to someone who was probably got laid during the recession and isn't around anymore, or some open source repo creator who is now being forced to work 18 hours a day by their new job and doesn't have the energy to update any code. It needs to slowly creep out and annoy other people nearby before someone will actually care. When the neighbors constantly have wailing noises coming out of their living room TV they will be incentivized to contribute to the repo.
    - Dylan16807 3y ago
      If you have the magic powers required to pull off the cat noise plan, you can do much better than keeping things around for 30 years. If you don't, warning and removing is most of the toolbox.
    - eru 3y ago
      > It should not be "removed" as a step function. It should get more annoying over time but still function for the next 30 years, at which point it's incredibly, incredibly annoying to call the function (but still works). Keeping stuff around costs maintenance effort. Python is mostly developed by volunteers (either directly by occasional volunteers like me, or by professionals who are paid by organisations like Microsoft, who volunteer the money.)
personjerry 3y ago
Who is updating to every 3.x?
- d1l 3y ago
  If you maintain a library you’ll hear about this kind of shit early and often.
- ddxv 3y ago
  I don't manage anything too crucial, but I like to keep every project I work on updated to the latest Python. I've noticed I'm often quite early with this, and help open issues with any libraries that are lagging. For example for Py3.12 I'd say about half of what I work on is on Python3.12 now. Slower projects are 3.11 and those I'm just a user of (like stable diffusion) are still on 3.10. I don't know of any projects I use that are less than 3.10. This also is in line with how I am around most tech. I like to keep all databases and libraries up to date for security reasons.
Clamchop 3y ago
A typical pattern is to deprecate in minor versions and remove in the next major version, encouraging gentle readiness for that major update. I don't know enough about Python to say this is what they're doing, but it fits in this case.
- OJFord 3y ago
  That's not what they do, in semver terms you get breaking changes in minor version bumps. To be fair, as a result there's maybe more resistance to making any such change than there otherwise would be, but in nicher standard lib modules there are API/semantic changes from one version to another. I'd prefer semver, like it sounds GP would, but failing that I'd prefer totally owning that the version is fairly meaningless, and doing something like 2023.x as pip does for example. At least then everyone knows they should check for anything like that. (Not to mention it carries the additional age/potentially out of date information.)
  - selcuka 3y ago
    > something like 2023.x as pip does Not sure what you mean by that. The version of pip currently installed on my laptop is 23.3.1. Did you mean some other package, such as pytz?
    - Mehdi2277 3y ago
      23 comes from 2023. Pip uses calendar based versioning. The 23.3.1 means 2023, 3rd quarter, second release. The last number is for bug fix release. The first two numbers are purely date based and do not follow semvar.
      - selcuka 3y ago
        D'oh. All those years I've never noticed that. Thanks for the info.
MattGaiser 3y ago
People are spoiled by Java, which never seems to break anything.
- Wowfunhappy 3y ago
  Are they spoiled, or do they have a reasonable expectation?
- StressedDev 3y ago
  Java gets this right. In general, breaking existing working software is a very bad idea and only should be done in extreme cases. One thing a lot of people over look is breaking changes make life easier for the few people who write the language, library, OS, database, etc. but make life much harder to the many people who maintain or use applications.
  - kelipso 3y ago
    Ya and it's multiple magnitudes more people who maintain or use applications. Every single breakage is a massive massive waste of people's time.
    - MiddleEndian 3y ago
      >Every single breakage is a massive massive waste of people's time. I think this applies to a ton of things. For instance, in 2015, my electricity provider switched from NSTAR to EverSource. For whatever reason they couldn't auto-migrate everyone's accounts. I ended up having to call them on the phone. Took maybe 20-30 minutes, I think. According to https://en.wikipedia.org/wiki/NSTAR_(company) https://en.wikipedia.org/wiki/NSTAR_(company) they had 1.4 million customers. Massive waste of time for a ton of people. Societally, there should be a way to deal with this. Obviously not possible for real, but maybe it would be appropriate lock the execs and board of directors in a room that just continuously plays hold music for a week so their time is wasted as well.
    - pjc50 3y ago
      This. This is why Windows has kept backwards compatibility by heroic efforts on the part of its devs: because they do actually have clients (US-DOD is the obvious one) who can refuse breaking updates entirely and have enough leverage to make that stick. They also know that users who do somehow leave the Windows desktop ecosystem aren't coming back. C# managed a semi-breaking migration by having a library format that was compatible with old and new: "netstandard".
cdogl 3y ago
The problem is open source. I generally agree with your analysis of the migration path, but who’s going to go through every open source library and do this grunt work? It comes at a huge cost however you swing it, and it’s not theoretical: I recall trying to switch to Python 3 over and over during those dark years only to find an essential library couldn’t support it.
- upbeat_general 3y ago
  Yeah this is the real reason right here. Updating your code: Easy. Updating 10 different libraries that you depend on but don't know anything about: Not so easy
Thaxll 3y ago
You forgot to mention when did you did it, back then a lot of librairies where not compatible.
Wowfunhappy 3y ago
> The people who flamed Python for these breaking changes were either extreme outliers or the sort of people who would be happy with the language never making breaking changes, ever, no matter how bad of mistakes in design were discovered. Count me in the "don't make breaking changes" camp. You can spit out warnings to prevent developers from using certain language features in new code, but changes which outright break old code should be extremely rare. This is how Javascript works (the language, not the ecosystem around it), and it means I can update my web browser and continue visiting old websites. I don't think most languages should go as far as Javascript. However, the case for a breaking change should be a heck of a lot stronger than "we think all timestamps should have timezones now." That's a fine opinion to have, but it doesn't mean you should go break stuff.
- derefr 3y ago
  > However, the case for a breaking change should be a heck of a lot stronger than "we think all timestamps should have timezones". How about if you rephrase the case as "we've sampled codebases that use naive timestamps, and at least 50% of them had critical bugs related to mistaken assumptions made about timezones by functions used to produce vs consume those timestamps. Your code is already broken. This update just makes the compiler force you to rewrite your code in a way that makes the errors obvious and/or into type errors — and, therefore, which forces you to fix those bugs."
  - Wowfunhappy 3y ago
    Presumably this code works for me if I'm using it. In most cases, an imperfect program I can run is better than one I can't run. Yes there are exceptions. In the ideal case, the author updates their code so it keeps working. But what if I'm not the author and know nothing about the codebase? Is forcing me to make a naïve change to get things working again more likely to remove bugs or add them? The compiler can warn me that I probably have a bug and should change my code. But it shouldn't hold the program hostage until I comply. This isn't a blanket statement in favor of permissive compilers. It's fine for the compiler to force good practices the first time. I just have a problem with the compiler going back and changing its mind later.
akra 3y ago
For Python that is properly source controlled, audited and understood I do agree it isn't that hard to port. However unlike many more compiled languages there's probably a LOT of python code out there that isn't which makes changes to Python and its APIs more risky. Don't get me wrong - this is one of the reasons why the language is popular - it is seen as "easy and approachable"; you just write a script and it works for the most part. It also was often the only language built into many distro's other than bash making it ultra convenient. Which means a lot of it written isn't catalogued, in source control, whatever but still doing stuff people need and often forgotten about. This makes changes of Python IMO more risky. I've seen Python scripts on old Linux images lurking in the wild working for even decades without people realizing they are there set up by non-dev's people back in the day. Another example would be some financial analyst coding a script to produce reports that are used to make decisions and need same behavior each time - one day the date time function maybe in this article will just stop working often with the original person who wrote it just to get stuff done quick long gone. IMO migration from 2 to 3 was particularly painful for Python because of the way it/was is used, historically who the main users of it, its popularity (wide blast radius) and how it is often Python is updated by blanket system upgrades given it is standard on many machines. Being aware of what needs to be upgraded for a given large scale environment is most of the work.
xorcist 3y ago
The only reason this is a potential source of bugs is because of the poorly thought out changes in Python 3. It is not a problem with Python 2 code, and it wouldn't have been a problem should there have been backwards compatible types in Python 3. What's done is done, and the best way to avoid this is to add support in linters to warn for this, or a runtime warning when that can be done safely. Deprecating or removing a method is madness for any language with any real world use. The only thing that leads to is a lot of unnecessary work for a lot of people, and there will be a best practice to save old versions of the interpreter to run old software.
stefantalpalaru 3y ago
[dead]
saaspirant 3y ago
Can you share some nuggets about how did you migrate a bigger codebase from 2 to 3 in less than 40 hours? I tried with a Django codebase and didn't know where to go.. 2to3 tools they provided didn't help much. Can you give steps involved? I'll dig deeper on each step. Thanks
- kerkeslager 3y ago
  In general the approach is pretty heavily dependent on a trustworthy test suite. If you don't have that, you'll have to write a lot of tests which will take more than 40 hours, but... Python is pretty heavily dependent on having a trustworthy test suite. All of the codebases I migrated were Django codebases. Basically, I would: 1. Upgrade to the latest 2.x, run the test suite and a bunch of manual tests, and fix any errors/warnings. 2. Upgrade all the dependencies one by one to versions that support version 3, run the test suite and the manual tests, and fix any errors/warnings. This was generally the hardest part, because some projects had imported a lot of bleeding edge nonsense which was abandoned before Python 3 support was implemented, so I had to a) encapsulate the dependency, and then b) rewrite the encapsulated functionality either with my own code, or with a more up-to-date library. This is a predictable result of importing nonsense, and I blame the developers who did that, not the Python team. 3. Upgrade to 3.x, run the test suite of manual tests, and fix any errors/warnings.
ensignavenger 3y ago
Deprecating is not removing. Deprecating a method breaks nothing. It just shows a deprecation warning when you use it and have deprecation warnings turned on.
- Kwpolska 3y ago
  A deprecation in Python will be followed by removal a few years from now (where a few ≈ 2 in recent practice).
  - mkesper 3y ago
    And that's a good thing! Do not hold on to broken cruft but give some time (more than just a few months (Ansible and other culprits) but definitely less tan 10 years (Python 3 transition)) for code maintainers to handle it.
    - larschdk 3y ago
      They should respect semantic versioning and ensure that any 3.x version remains backwards compatible with 3.0 code.
      - starlevel003 3y ago
        Python does not use semantic versioning and never has.
    - Kwpolska 3y ago
      Removing “broken cruft” from a programming language’s standard library has a very high cost associated with it. If you look at Java, the list of things removed in the past decade is quite short [0], and the removed things probably weren’t super important or popular. Python’s deletionists go too far IMO. [0] https://javaalmanac.io/jdk/17/apidiff/8/ https://javaalmanac.io/jdk/17/apidiff/8/
logicchains 3y ago
>As a contractor, I singlehandedly migrated 4 different codebases from 2 to 3; a total of just under a million lines of code, and none of these codebases took more than 40 hours. This was not some horrific burden for most companies using Python. The people who flamed Python for these breaking changes were either extreme outliers or the sort of people who would be happy with the language never making breaking changes, ever, no matter how bad of mistakes in design were discovered. Imagine I just randomly came up to you and demanded a few thousand dollars, you probably wouldn't be too happy about it. That's essentially what the python maintainers did: collectively demand tens of thousands of hours worth of labor from developers, labor that could have been spent doing something enjoyable or profitable, but instead was wasted on menial work because the Python developers thought some minor changes were worth breaking things for everyone. I.e. they put a much higher value on their own subjective ideas of cleanliness than on the time of all Python users.
- cgeier 3y ago
  Alternatively, users can just stay at the python version they wrote the code for.
  - rightbyte 3y ago
    Which they did in many cases.
    - deleted 3y ago
      [deleted]
    - bdowling 3y ago
      Which they still do in many, many cases.
- bbkane 3y ago
  I think it'd be more like if you gave me a car, and then, years later, you said you needed the car to build a plane for me. It would screw up my plans, but I wouldn't assign moral blame to you. I'd say "thanks for the car while I had it, I'll find another car vendor"
eternityforest 3y ago
I am not a fan of all these breaking changes. I still love Python, but it's just too much with these tiny tweaks. Developers really seem to like endless polishing, even when more important stuff still isn't done. On the other hand... Maintenance is a thing. Less code to maintain means less work. I can see why they want to get rid of stuff, it shouldn't be too hard to update, it's just annoying. Can't complain too much, at least it's not like C, where the big dependencies seem to like making such big changes it takes years to port.
sproketboy 3y ago
[dead]
gymbeaux 3y ago
What most people don’t seem to realize is that there aren’t only language changes to contend with, but framework changes as well. This line is further blurred in language-framework pairs like C# and .NET, where (as a web dev) you’re pretty much always going to be using .NET if you’re using C#, whether you realize it or not. In these cases, framework revisions are what getcha, not language changes. Microsoft does a good job of maintaining backwards compatibility as C# the language changes, but framework upgrades (eg .NET Core 3.1 to .NET 5) will absolutely cause headache via runtime exceptions, etc. So yeah Python ain’t that bad. I can’t speak to frameworks like Django, though.
- dfinninger 3y ago
  Django was actually fine. They had the same codebase running on Python 2 and three, so you could update whenever. Since Django includes so many batteries, I didn’t have any trouble porting that over. However, early versions of Python 3 were slower than Python 2, and also some breaking changes were getting rolled back (e.g. PEP-414, which was targeting Python 3.3), which contributed to a lot of library authors dragging their feet in upgrading their support. So, yes, it was libraries causing the most headaches, but there was a sense at the time of wondering when the upgrade would become “real”. Depreciating py2 took 11 years after the release of 3.0.

dheera 3y ago

Deprecation shouldn't break peoples' code all of a sudden. Code should still work, and just get more annoying over time. Initially, spit out warnings that deprecation is going to happen. Then, start changing the user's desktop background. Then, start playing nice cat meowing sounds out of the speaker. Then, start playing ghastly wailing sounds out of the speaker. Then, start sending e-mails. Then, scan for IoT light bulbs and mess with the colors. Then, scan for bluetooth headsets within a 50 meter radius and play wailing noises out of those. Then, impose a 0.001 second delay. Then a 0.01 second delay. Then a 0.1 second delay. Then a 1.0 second delay. Then start scraping LinkedIn for engineer resumes and e-mail them to any sysadmin e-mails you can find in the various config files in /etc/ and /home/. Don't suddenly deprecate stuff and break people's code on a moment's notice. Most likely the person who wrote the code using the deprecated function was laid off 2 years ago and the company still depends on it.

panarky 3y ago
> Don't suddenly deprecate stuff and break people's code on a moment's notice. "Deprecated" doesn't mean "break people's code on a moment's notice". FTA: "datetime.datetime’s utcnow() and utcfromtimestamp() are deprecated and will be removed in a future version" "likely a few years out" "once you switch to Python 3.12 or newer you will start seeing deprecation messages on your console and your logs, and these can get annoying"
- dheera 3y ago
  > will be removed in a future version It should not be "removed" as a step function. It should get more annoying over time but still function for the next 30 years, at which point it's incredibly, incredibly annoying to call the function (but still works). This helps products continue running but simultaneously incentivizes the people on top to hire engineers to make the code less annoying without completely wrecking the business or downstream software that depends on it. Basically, the servers will still run and the customers will be happy but if the VP sitting 50m from the servers doesn't want cat noises coming out of their headphones they need to hire an engineer ASAP or it will turn into wailing noises pretty soon. > you will start seeing deprecation messages on your console and your logs, and these can get annoying Not annoying enough. This only sends messages to someone who was probably got laid during the recession and isn't around anymore, or some open source repo creator who is now being forced to work 18 hours a day by their new job and doesn't have the energy to update any code. It needs to slowly creep out and annoy other people nearby before someone will actually care. When the neighbors constantly have wailing noises coming out of their living room TV they will be incentivized to contribute to the repo.
  - Dylan16807 3y ago
    If you have the magic powers required to pull off the cat noise plan, you can do much better than keeping things around for 30 years. If you don't, warning and removing is most of the toolbox.
  - eru 3y ago
    > It should not be "removed" as a step function. It should get more annoying over time but still function for the next 30 years, at which point it's incredibly, incredibly annoying to call the function (but still works). Keeping stuff around costs maintenance effort. Python is mostly developed by volunteers (either directly by occasional volunteers like me, or by professionals who are paid by organisations like Microsoft, who volunteer the money.)

personjerry 3y ago

Who is updating to every 3.x?

d1l 3y ago
If you maintain a library you’ll hear about this kind of shit early and often.
ddxv 3y ago
I don't manage anything too crucial, but I like to keep every project I work on updated to the latest Python. I've noticed I'm often quite early with this, and help open issues with any libraries that are lagging. For example for Py3.12 I'd say about half of what I work on is on Python3.12 now. Slower projects are 3.11 and those I'm just a user of (like stable diffusion) are still on 3.10. I don't know of any projects I use that are less than 3.10. This also is in line with how I am around most tech. I like to keep all databases and libraries up to date for security reasons.

Clamchop 3y ago

A typical pattern is to deprecate in minor versions and remove in the next major version, encouraging gentle readiness for that major update. I don't know enough about Python to say this is what they're doing, but it fits in this case.

OJFord 3y ago
That's not what they do, in semver terms you get breaking changes in minor version bumps. To be fair, as a result there's maybe more resistance to making any such change than there otherwise would be, but in nicher standard lib modules there are API/semantic changes from one version to another. I'd prefer semver, like it sounds GP would, but failing that I'd prefer totally owning that the version is fairly meaningless, and doing something like 2023.x as pip does for example. At least then everyone knows they should check for anything like that. (Not to mention it carries the additional age/potentially out of date information.)
- selcuka 3y ago
  > something like 2023.x as pip does Not sure what you mean by that. The version of pip currently installed on my laptop is 23.3.1. Did you mean some other package, such as pytz?
  - Mehdi2277 3y ago
    23 comes from 2023. Pip uses calendar based versioning. The 23.3.1 means 2023, 3rd quarter, second release. The last number is for bug fix release. The first two numbers are purely date based and do not follow semvar.
    - selcuka 3y ago
      D'oh. All those years I've never noticed that. Thanks for the info.

MattGaiser 3y ago

People are spoiled by Java, which never seems to break anything.

Wowfunhappy 3y ago
Are they spoiled, or do they have a reasonable expectation?
StressedDev 3y ago
Java gets this right. In general, breaking existing working software is a very bad idea and only should be done in extreme cases. One thing a lot of people over look is breaking changes make life easier for the few people who write the language, library, OS, database, etc. but make life much harder to the many people who maintain or use applications.
- kelipso 3y ago
  Ya and it's multiple magnitudes more people who maintain or use applications. Every single breakage is a massive massive waste of people's time.
  - MiddleEndian 3y ago
    >Every single breakage is a massive massive waste of people's time. I think this applies to a ton of things. For instance, in 2015, my electricity provider switched from NSTAR to EverSource. For whatever reason they couldn't auto-migrate everyone's accounts. I ended up having to call them on the phone. Took maybe 20-30 minutes, I think. According to https://en.wikipedia.org/wiki/NSTAR_(company) https://en.wikipedia.org/wiki/NSTAR_(company) they had 1.4 million customers. Massive waste of time for a ton of people. Societally, there should be a way to deal with this. Obviously not possible for real, but maybe it would be appropriate lock the execs and board of directors in a room that just continuously plays hold music for a week so their time is wasted as well.
  - pjc50 3y ago
    This. This is why Windows has kept backwards compatibility by heroic efforts on the part of its devs: because they do actually have clients (US-DOD is the obvious one) who can refuse breaking updates entirely and have enough leverage to make that stick. They also know that users who do somehow leave the Windows desktop ecosystem aren't coming back. C# managed a semi-breaking migration by having a library format that was compatible with old and new: "netstandard".

cdogl 3y ago

The problem is open source. I generally agree with your analysis of the migration path, but who’s going to go through every open source library and do this grunt work? It comes at a huge cost however you swing it, and it’s not theoretical: I recall trying to switch to Python 3 over and over during those dark years only to find an essential library couldn’t support it.

upbeat_general 3y ago
Yeah this is the real reason right here. Updating your code: Easy. Updating 10 different libraries that you depend on but don't know anything about: Not so easy

Thaxll 3y ago

You forgot to mention when did you did it, back then a lot of librairies where not compatible.

Wowfunhappy 3y ago

> The people who flamed Python for these breaking changes were either extreme outliers or the sort of people who would be happy with the language never making breaking changes, ever, no matter how bad of mistakes in design were discovered. Count me in the "don't make breaking changes" camp. You can spit out warnings to prevent developers from using certain language features in new code, but changes which outright break old code should be extremely rare. This is how Javascript works (the language, not the ecosystem around it), and it means I can update my web browser and continue visiting old websites. I don't think most languages should go as far as Javascript. However, the case for a breaking change should be a heck of a lot stronger than "we think all timestamps should have timezones now." That's a fine opinion to have, but it doesn't mean you should go break stuff.

derefr 3y ago
> However, the case for a breaking change should be a heck of a lot stronger than "we think all timestamps should have timezones". How about if you rephrase the case as "we've sampled codebases that use naive timestamps, and at least 50% of them had critical bugs related to mistaken assumptions made about timezones by functions used to produce vs consume those timestamps. Your code is already broken. This update just makes the compiler force you to rewrite your code in a way that makes the errors obvious and/or into type errors — and, therefore, which forces you to fix those bugs."
- Wowfunhappy 3y ago
  Presumably this code works for me if I'm using it. In most cases, an imperfect program I can run is better than one I can't run. Yes there are exceptions. In the ideal case, the author updates their code so it keeps working. But what if I'm not the author and know nothing about the codebase? Is forcing me to make a naïve change to get things working again more likely to remove bugs or add them? The compiler can warn me that I probably have a bug and should change my code. But it shouldn't hold the program hostage until I comply. This isn't a blanket statement in favor of permissive compilers. It's fine for the compiler to force good practices the first time. I just have a problem with the compiler going back and changing its mind later.

akra 3y ago

For Python that is properly source controlled, audited and understood I do agree it isn't that hard to port. However unlike many more compiled languages there's probably a LOT of python code out there that isn't which makes changes to Python and its APIs more risky. Don't get me wrong - this is one of the reasons why the language is popular - it is seen as "easy and approachable"; you just write a script and it works for the most part. It also was often the only language built into many distro's other than bash making it ultra convenient. Which means a lot of it written isn't catalogued, in source control, whatever but still doing stuff people need and often forgotten about. This makes changes of Python IMO more risky. I've seen Python scripts on old Linux images lurking in the wild working for even decades without people realizing they are there set up by non-dev's people back in the day. Another example would be some financial analyst coding a script to produce reports that are used to make decisions and need same behavior each time - one day the date time function maybe in this article will just stop working often with the original person who wrote it just to get stuff done quick long gone. IMO migration from 2 to 3 was particularly painful for Python because of the way it/was is used, historically who the main users of it, its popularity (wide blast radius) and how it is often Python is updated by blanket system upgrades given it is standard on many machines. Being aware of what needs to be upgraded for a given large scale environment is most of the work.

xorcist 3y ago

The only reason this is a potential source of bugs is because of the poorly thought out changes in Python 3. It is not a problem with Python 2 code, and it wouldn't have been a problem should there have been backwards compatible types in Python 3. What's done is done, and the best way to avoid this is to add support in linters to warn for this, or a runtime warning when that can be done safely. Deprecating or removing a method is madness for any language with any real world use. The only thing that leads to is a lot of unnecessary work for a lot of people, and there will be a best practice to save old versions of the interpreter to run old software.

stefantalpalaru 3y ago

[dead]

saaspirant 3y ago

Can you share some nuggets about how did you migrate a bigger codebase from 2 to 3 in less than 40 hours? I tried with a Django codebase and didn't know where to go.. 2to3 tools they provided didn't help much. Can you give steps involved? I'll dig deeper on each step. Thanks

kerkeslager 3y ago
In general the approach is pretty heavily dependent on a trustworthy test suite. If you don't have that, you'll have to write a lot of tests which will take more than 40 hours, but... Python is pretty heavily dependent on having a trustworthy test suite. All of the codebases I migrated were Django codebases. Basically, I would: 1. Upgrade to the latest 2.x, run the test suite and a bunch of manual tests, and fix any errors/warnings. 2. Upgrade all the dependencies one by one to versions that support version 3, run the test suite and the manual tests, and fix any errors/warnings. This was generally the hardest part, because some projects had imported a lot of bleeding edge nonsense which was abandoned before Python 3 support was implemented, so I had to a) encapsulate the dependency, and then b) rewrite the encapsulated functionality either with my own code, or with a more up-to-date library. This is a predictable result of importing nonsense, and I blame the developers who did that, not the Python team. 3. Upgrade to 3.x, run the test suite of manual tests, and fix any errors/warnings.

ensignavenger 3y ago

Deprecating is not removing. Deprecating a method breaks nothing. It just shows a deprecation warning when you use it and have deprecation warnings turned on.

Kwpolska 3y ago
A deprecation in Python will be followed by removal a few years from now (where a few ≈ 2 in recent practice).
- mkesper 3y ago
  And that's a good thing! Do not hold on to broken cruft but give some time (more than just a few months (Ansible and other culprits) but definitely less tan 10 years (Python 3 transition)) for code maintainers to handle it.
  - larschdk 3y ago
    They should respect semantic versioning and ensure that any 3.x version remains backwards compatible with 3.0 code.
    - starlevel003 3y ago
      Python does not use semantic versioning and never has.
  - Kwpolska 3y ago
    Removing “broken cruft” from a programming language’s standard library has a very high cost associated with it. If you look at Java, the list of things removed in the past decade is quite short [0], and the removed things probably weren’t super important or popular. Python’s deletionists go too far IMO. [0] https://javaalmanac.io/jdk/17/apidiff/8/ https://javaalmanac.io/jdk/17/apidiff/8/

logicchains 3y ago

>As a contractor, I singlehandedly migrated 4 different codebases from 2 to 3; a total of just under a million lines of code, and none of these codebases took more than 40 hours. This was not some horrific burden for most companies using Python. The people who flamed Python for these breaking changes were either extreme outliers or the sort of people who would be happy with the language never making breaking changes, ever, no matter how bad of mistakes in design were discovered. Imagine I just randomly came up to you and demanded a few thousand dollars, you probably wouldn't be too happy about it. That's essentially what the python maintainers did: collectively demand tens of thousands of hours worth of labor from developers, labor that could have been spent doing something enjoyable or profitable, but instead was wasted on menial work because the Python developers thought some minor changes were worth breaking things for everyone. I.e. they put a much higher value on their own subjective ideas of cleanliness than on the time of all Python users.

cgeier 3y ago
Alternatively, users can just stay at the python version they wrote the code for.
- rightbyte 3y ago
  Which they did in many cases.
  - deleted 3y ago
    [deleted]
  - bdowling 3y ago
    Which they still do in many, many cases.
bbkane 3y ago
I think it'd be more like if you gave me a car, and then, years later, you said you needed the car to build a plane for me. It would screw up my plans, but I wouldn't assign moral blame to you. I'd say "thanks for the car while I had it, I'll find another car vendor"

eternityforest 3y ago

I am not a fan of all these breaking changes. I still love Python, but it's just too much with these tiny tweaks. Developers really seem to like endless polishing, even when more important stuff still isn't done. On the other hand... Maintenance is a thing. Less code to maintain means less work. I can see why they want to get rid of stuff, it shouldn't be too hard to update, it's just annoying. Can't complain too much, at least it's not like C, where the big dependencies seem to like making such big changes it takes years to port.

sproketboy 3y ago

[dead]

gymbeaux 3y ago

What most people don’t seem to realize is that there aren’t only language changes to contend with, but framework changes as well. This line is further blurred in language-framework pairs like C# and .NET, where (as a web dev) you’re pretty much always going to be using .NET if you’re using C#, whether you realize it or not. In these cases, framework revisions are what getcha, not language changes. Microsoft does a good job of maintaining backwards compatibility as C# the language changes, but framework upgrades (eg .NET Core 3.1 to .NET 5) will absolutely cause headache via runtime exceptions, etc. So yeah Python ain’t that bad. I can’t speak to frameworks like Django, though.

dfinninger 3y ago
Django was actually fine. They had the same codebase running on Python 2 and three, so you could update whenever. Since Django includes so many batteries, I didn’t have any trouble porting that over. However, early versions of Python 3 were slower than Python 2, and also some breaking changes were getting rolled back (e.g. PEP-414, which was targeting Python 3.3), which contributed to a lot of library authors dragging their feet in upgrading their support. So, yes, it was libraries causing the most headaches, but there was a sense at the time of wondering when the upgrade would become “real”. Depreciating py2 took 11 years after the release of 3.0.